Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RVV] CVA6 re-parametrization and MMU interface #2652

Draft
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

mp-17
Copy link
Contributor

@mp-17 mp-17 commented Dec 4, 2024

Draft PR as a follow-up to the discussion on extending Linux support to the Ara vector processor.

Addition:

  • Add external MMU interface to share the MMU with the external accelerator.

Fixes:

  • Guard some X-IF code lines with correct parameter in cva6.sv.
  • Parametrize the tracer interface with NrCommitPorts.

Modification:

  • Move the parameters and data types to be shared with Ara into the cva6 config package.
  • 2 commit ports by default in cv64a6_imafdcv_config_pkg.
  • Move exception_t definition in the config packages.
  • Use spill register in acc_dispatcher to fully decouple timing.
  • Guard some icache lines with FETCH_USER_EN.

Todo:

  • Clean with Verible
  • Test this rebased version with Ara
  • Add the definition of exception_t, PLEN, GPLEN, PPNW, TRANS_ID_BITS to all the other config packages.

Questions:

  • Should exception_t remain as a parameter in cva6.sv's interface?

parameter type acc_resp_t = logic,
parameter type accelerator_req_t = logic,
parameter type accelerator_resp_t = logic,
parameter type acc_mmu_req_t = logic,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
parameter type acc_mmu_req_t = logic,
parameter type acc_mmu_req_t = logic,

Comment on lines 210 to 211
logic acc_req_valid;
logic acc_req_ready;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
logic acc_req_valid;
logic acc_req_ready;
logic acc_req_valid;
logic acc_req_ready;

Comment on lines 217 to 224
.clk_i (clk_i),
.rst_ni (rst_ni),
.clr_i (1'b0),
.testmode_i(1'b0),
.data_i (acc_req),
.valid_i (acc_req_valid),
.ready_o (acc_req_ready),
.data_o (acc_req_int),
.valid_o (acc_req_o.req_valid),
.ready_i (acc_resp_i.req_ready)
.valid_o (acc_req_o.acc_req.req_valid),
.ready_i (acc_resp_i.acc_resp.req_ready)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
.clk_i (clk_i),
.rst_ni (rst_ni),
.clr_i (1'b0),
.testmode_i(1'b0),
.data_i (acc_req),
.valid_i (acc_req_valid),
.ready_o (acc_req_ready),
.data_o (acc_req_int),
.valid_o (acc_req_o.req_valid),
.ready_i (acc_resp_i.req_ready)
.valid_o (acc_req_o.acc_req.req_valid),
.ready_i (acc_resp_i.acc_resp.req_ready)
.clk_i (clk_i),
.rst_ni (rst_ni),
.data_i (acc_req),
.valid_i(acc_req_valid),
.ready_o(acc_req_ready),
.data_o (acc_req_int),
.valid_o(acc_req_o.acc_req.req_valid),
.ready_i(acc_resp_i.acc_resp.req_ready)

Comment on lines 237 to 238
assign acc_req_o.acc_mmu_resp = acc_mmu_resp_i;
assign acc_req_o.acc_mmu_en = acc_mmu_en_i;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
assign acc_req_o.acc_mmu_resp = acc_mmu_resp_i;
assign acc_req_o.acc_mmu_en = acc_mmu_en_i;
assign acc_req_o.acc_mmu_resp = acc_mmu_resp_i;
assign acc_req_o.acc_mmu_en = acc_mmu_en_i;

Comment on lines 282 to 285
assign acc_trans_id_o = acc_resp_i.acc_resp.trans_id;
assign acc_result_o = acc_resp_i.acc_resp.result;
assign acc_valid_o = acc_resp_i.acc_resp.resp_valid;
assign acc_exception_o = acc_resp_i.acc_resp.exception;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
assign acc_trans_id_o = acc_resp_i.acc_resp.trans_id;
assign acc_result_o = acc_resp_i.acc_resp.result;
assign acc_valid_o = acc_resp_i.acc_resp.resp_valid;
assign acc_exception_o = acc_resp_i.acc_resp.exception;
assign acc_trans_id_o = acc_resp_i.acc_resp.trans_id;
assign acc_result_o = acc_resp_i.acc_resp.result;
assign acc_valid_o = acc_resp_i.acc_resp.resp_valid;
assign acc_exception_o = acc_resp_i.acc_resp.exception;

end
end
end

if (CVA6Cfg.EnableAccelerator) begin
// The MMU can be connected to CVA6 or the ACCELERATOR
enum logic {CVA6, ACC} mmu_state_d, mmu_state_q;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
enum logic {CVA6, ACC} mmu_state_d, mmu_state_q;
enum logic {
CVA6,
ACC
}
mmu_state_d, mmu_state_q;

// This logic can be optimized to reduce answer latency and contention
always_comb begin
// Maintain state
mmu_state_d = mmu_state_q;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
mmu_state_d = mmu_state_q;
mmu_state_d = mmu_state_q;

assign cva6_dtlb_hit = dtlb_hit;
assign cva6_dtlb_ppn = dtlb_ppn;
// No accelerator
assign acc_mmu_resp_o = '0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
assign acc_mmu_resp_o = '0;
assign acc_mmu_resp_o = '0;

// No accelerator
assign acc_mmu_resp_o = '0;
// Feed forward the lsu_ctrl bypass
assign lsu_ctrl = lsu_ctrl_byp;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
assign lsu_ctrl = lsu_ctrl_byp;
assign lsu_ctrl = lsu_ctrl_byp;

Comment on lines 555 to 618
ld_valid_i = 1'b0;
st_valid_i = 1'b0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
ld_valid_i = 1'b0;
st_valid_i = 1'b0;
ld_valid_i = 1'b0;
st_valid_i = 1'b0;

Copy link
Contributor

github-actions bot commented Dec 4, 2024

❌ failed run, report available here.

@JeanRochCoulon
Copy link
Contributor

Hello @mp-17 To solve the Verible format issues, you can execute the Verible command from https://github.com/openhwgroup/cva6/blob/master/CONTRIBUTING.md

@mp-17
Copy link
Contributor Author

mp-17 commented Dec 5, 2024

Hello @JeanRochCoulon, thanks, I will! The PR is still work in progress, I put it here as a placeholder. I will rework it today and ping you when it's ready. Don't spend time on the diff now, it's full of unrelated stuff :-)

Comment on lines 297 to 298
assign acc_ld_disp = acc_req_valid && (acc_insn_queue_o.operation == ACCEL_OP_LOAD);
assign acc_st_disp = acc_req_valid && (acc_insn_queue_o.operation == ACCEL_OP_STORE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
assign acc_ld_disp = acc_req_valid && (acc_insn_queue_o.operation == ACCEL_OP_LOAD);
assign acc_st_disp = acc_req_valid && (acc_insn_queue_o.operation == ACCEL_OP_STORE);
assign acc_ld_disp = acc_req_valid && (acc_insn_queue_o.operation == ACCEL_OP_LOAD);
assign acc_st_disp = acc_req_valid && (acc_insn_queue_o.operation == ACCEL_OP_STORE);

Comment on lines 215 to 240
logic [ 31:0] mmu_tinst;
logic mmu_hs_ld_st_inst;
logic mmu_hlvx_inst;
exception_t mmu_exception, cva6_mmu_exception, acc_mmu_exception;
exception_t pmp_exception;
icache_areq_t pmp_icache_areq_i;
logic pmp_translation_valid;
logic dtlb_hit, cva6_dtlb_hit, acc_dtlb_hit;
logic [ CVA6Cfg.PPNW-1:0] dtlb_ppn, cva6_dtlb_ppn, acc_dtlb_ppn;

logic ld_valid;
logic [CVA6Cfg.TRANS_ID_BITS-1:0] ld_trans_id;
logic [ CVA6Cfg.XLEN-1:0] ld_result;
logic st_valid;
logic [CVA6Cfg.TRANS_ID_BITS-1:0] st_trans_id;
logic [ CVA6Cfg.XLEN-1:0] st_result;

logic [ 11:0] page_offset;
logic page_offset_matches;

exception_t misaligned_exception, cva6_misaligned_exception, acc_misaligned_exception;
exception_t ld_ex;
exception_t st_ex;

logic hs_ld_st_inst;
logic hlvx_inst;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
logic [ 31:0] mmu_tinst;
logic mmu_hs_ld_st_inst;
logic mmu_hlvx_inst;
exception_t mmu_exception, cva6_mmu_exception, acc_mmu_exception;
exception_t pmp_exception;
icache_areq_t pmp_icache_areq_i;
logic pmp_translation_valid;
logic dtlb_hit, cva6_dtlb_hit, acc_dtlb_hit;
logic [ CVA6Cfg.PPNW-1:0] dtlb_ppn, cva6_dtlb_ppn, acc_dtlb_ppn;
logic ld_valid;
logic [CVA6Cfg.TRANS_ID_BITS-1:0] ld_trans_id;
logic [ CVA6Cfg.XLEN-1:0] ld_result;
logic st_valid;
logic [CVA6Cfg.TRANS_ID_BITS-1:0] st_trans_id;
logic [ CVA6Cfg.XLEN-1:0] st_result;
logic [ 11:0] page_offset;
logic page_offset_matches;
exception_t misaligned_exception, cva6_misaligned_exception, acc_misaligned_exception;
exception_t ld_ex;
exception_t st_ex;
logic hs_ld_st_inst;
logic hlvx_inst;
logic [31:0] mmu_tinst;
logic mmu_hs_ld_st_inst;
logic mmu_hlvx_inst;
exception_t mmu_exception, cva6_mmu_exception, acc_mmu_exception;
exception_t pmp_exception;
icache_areq_t pmp_icache_areq_i;
logic pmp_translation_valid;
logic dtlb_hit, cva6_dtlb_hit, acc_dtlb_hit;
logic [CVA6Cfg.PPNW-1:0] dtlb_ppn, cva6_dtlb_ppn, acc_dtlb_ppn;
logic ld_valid;
logic [CVA6Cfg.TRANS_ID_BITS-1:0] ld_trans_id;
logic [ CVA6Cfg.XLEN-1:0] ld_result;
logic st_valid;
logic [CVA6Cfg.TRANS_ID_BITS-1:0] st_trans_id;
logic [ CVA6Cfg.XLEN-1:0] st_result;
logic [ 11:0] page_offset;
logic page_offset_matches;
exception_t misaligned_exception, cva6_misaligned_exception, acc_misaligned_exception;
exception_t ld_ex;
exception_t st_ex;
logic hs_ld_st_inst;
logic hlvx_inst;

Copy link
Contributor

github-actions bot commented Dec 5, 2024

❌ failed run, report available here.

Comment on lines +94 to +95
function automatic logic [CVA6Cfg.PLEN-1:0] paddrSizeAlign(
input logic [CVA6Cfg.PLEN-1:0] paddr, input logic [2:0] size);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
function automatic logic [CVA6Cfg.PLEN-1:0] paddrSizeAlign(
input logic [CVA6Cfg.PLEN-1:0] paddr, input logic [2:0] size);
function automatic logic [CVA6Cfg.PLEN-1:0] paddrSizeAlign(input logic [CVA6Cfg.PLEN-1:0] paddr,
input logic [2:0] size);

Comment on lines +139 to +140
input logic [CVA6Cfg.XLEN-1:0] data,
input logic [CVA6Cfg.XLEN_ALIGN_BYTES-1:0] offset, input logic [1:0] size);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
input logic [CVA6Cfg.XLEN-1:0] data,
input logic [CVA6Cfg.XLEN_ALIGN_BYTES-1:0] offset, input logic [1:0] size);
input logic [CVA6Cfg.XLEN-1:0] data, input logic [CVA6Cfg.XLEN_ALIGN_BYTES-1:0] offset,
input logic [1:0] size);

Comment on lines +152 to +153
input logic [CVA6Cfg.XLEN-1:0] data,
input logic [CVA6Cfg.XLEN_ALIGN_BYTES-1:0] offset, input logic [1:0] size);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
input logic [CVA6Cfg.XLEN-1:0] data,
input logic [CVA6Cfg.XLEN_ALIGN_BYTES-1:0] offset, input logic [1:0] size);
input logic [CVA6Cfg.XLEN-1:0] data, input logic [CVA6Cfg.XLEN_ALIGN_BYTES-1:0] offset,
input logic [1:0] size);

Comment on lines +97 to +98
logic [CVA6Cfg.SharedTlbDepth-1:0][SHARED_TLB_WAYS-1:0]
shared_tag_valid_q, shared_tag_valid_d;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
logic [CVA6Cfg.SharedTlbDepth-1:0][SHARED_TLB_WAYS-1:0]
shared_tag_valid_q, shared_tag_valid_d;
logic [CVA6Cfg.SharedTlbDepth-1:0][SHARED_TLB_WAYS-1:0] shared_tag_valid_q, shared_tag_valid_d;

Comment on lines +126 to +127
logic [CVA6Cfg.PtLevels+HYP_EXT-1:0][(CVA6Cfg.VpnLen/CVA6Cfg.PtLevels)-1:0]
vpn_d, vpn_q;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
logic [CVA6Cfg.PtLevels+HYP_EXT-1:0][(CVA6Cfg.VpnLen/CVA6Cfg.PtLevels)-1:0]
vpn_d, vpn_q;
logic [CVA6Cfg.PtLevels+HYP_EXT-1:0][(CVA6Cfg.VpnLen/CVA6Cfg.PtLevels)-1:0] vpn_d, vpn_q;

Comment on lines +130 to +131
logic [CVA6Cfg.INSTR_PER_FETCH-1:0]
rvc_branch, rvc_jump, rvc_jr, rvc_return, rvc_jalr, rvc_call;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
logic [CVA6Cfg.INSTR_PER_FETCH-1:0]
rvc_branch, rvc_jump, rvc_jr, rvc_return, rvc_jalr, rvc_call;
logic [CVA6Cfg.INSTR_PER_FETCH-1:0] rvc_branch, rvc_jump, rvc_jr, rvc_return, rvc_jalr, rvc_call;

Comment on lines +21 to +22
static function logic [PMP_LEN-1:0] base_to_conf(logic [WIDTH-1:0] base,
int unsigned size_i);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
static function logic [PMP_LEN-1:0] base_to_conf(logic [WIDTH-1:0] base,
int unsigned size_i);
static function logic [PMP_LEN-1:0] base_to_conf(logic [WIDTH-1:0] base, int unsigned size_i);

Comment on lines +114 to +115
logic [CVA6Cfg.NrCommitPorts-1:0][CVA6Cfg.TRANS_ID_BITS-1:0]
commit_pointer_n, commit_pointer_q;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[verible-verilog-format] reported by reviewdog 🐶

Suggested change
logic [CVA6Cfg.NrCommitPorts-1:0][CVA6Cfg.TRANS_ID_BITS-1:0]
commit_pointer_n, commit_pointer_q;
logic [CVA6Cfg.NrCommitPorts-1:0][CVA6Cfg.TRANS_ID_BITS-1:0] commit_pointer_n, commit_pointer_q;

@mp-17
Copy link
Contributor Author

mp-17 commented Dec 16, 2024

Hi @JeanRochCoulon, I cleaned up the code diff so that we can start discussing how to integrate the interface and re-parametrization modifications.
Let me know what you think of this PR or if you need additional refactoring or explanations before reviewing.

I have added additional localparams (such as PLEN) to the cv64a6_imafdcv_sv39_config_pkg.sv user config package. Before adapting the other packages too (to fix the CI), let me know if this way of handling the issue is okay or if we need to refine it.

Copy link
Contributor

❌ failed run, report available here.

@JeanRochCoulon
Copy link
Contributor

@cathales You should have suggestions to provide. @mp-17 tries to declare parameters needed for CV-X-IF/ARA integration.

Copy link
Contributor

@cathales cathales left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that you add fields to cva6_user_cfg_t not because you need to assign specific values to them, but because you want to access them in your configuration package.
This will impact all configurations and imply repetition.

Instead, you could call build_config once to be able to have access to the calculated values.

localparam cfg = build_config(cva6_cfg);

However, this would prevent from creating heterogeneous designs.

If you have one top-level module above cva6, I advise you to not add types into cv64a6_imafdcv_sv39_config_pkg. Instead, you can add them as localparam right into your top-level module and propagate them to cva6 and other modules.
To access to all calculated values, first add at the beginning of your top-level module:

    parameter config_pkg::cva6_cfg_t CVA6Cfg = build_config_pkg::build_config(
        cva6_config_pkg::cva6_cfg
    ),

This is how it is done in cva6.sv.

Also, when you instantiate CVA6, you can add cva6 #(.CVA6Cfg(CVA6Cfg)) parameter to re-use the configuration that you already calculated instead of implicitly calling build_config again to build the default value. This is what allows heterogeneous designs.

If you need several top level modules, we have another solution that we use for RVFI, just ask me if you need it.

PS: These modifications seem related to accelerator port, what about using CV-X-IF instead?

@mp-17
Copy link
Contributor Author

mp-17 commented Dec 18, 2024

Hello @cathales,

Thanks for the reply!

  1. I think you interpreted it correctly. I want to have a single source of truth for data types (e.g., exception_t, interfaces) and parameters so that I can reuse them both in CVA6 and in the tightly-coupled accelerator.
    From my understanding, you would prefer all these parameters to be defined in my top level only and then assigned as parameters to CVA6 and the accelerator.
    I will clean up the cv64a6_imafdcv_sv39_config_pkg.sv and try as you suggested. I will tell you if I meet any blockers.

  2. CV-X-IF: The plan discussed during our last joint meeting is to merge these custom interface modifications first and then discuss how to transition to the CV-X-IF later. This is also because CV-X-IF would need a custom memory interface anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants